A server is a network connected machine that performs some set of services or operations on behalf of one or more clients. A configuration loaded into server memory controls the services and operations performed by the server. The configuration specifies the operational logic for the server. The configuration can be used to transform a server into a content caching and distribution server, a billing server, a reporting server, and a firewall as some examples. More specifically, a configuration can set the caching policies, purging policies, access restrictions, quality of service, and other operational parameters for a server operationally defined as a content caching and distribution server.
A multi-tenant platform includes one or more servers that perform services and operations on behalf of different customers. Content delivery networks (CDNs), streaming providers, hosting services, and cloud service providers are examples of different multi-tenant platforms in which different customers rely on the same set of servers for services or content distribution.
In some multi-tenant platforms, the customers define configurations that control the services and operations performed by the servers. In some other multi-tenant platforms, the servers perform the same base set of services and operations (e.g., content delivery), but customer configurations customize various parameters or features of those services and operations. Stated differently, the customer configurations can involve high-level or low-level changes to the server operational logic. With respect to a content caching and distribution server, the configuration high-level changes could include customizing content type or file extensions to cache, and the configuration low-level changes could include changing the underlying caching policy implemented by the content caching and distribution server.
Multi-tenant platforms can grow to support thousands of customers. Each customer can define a different configuration. Each additional customer configuration places addition strain on the platform servers not only because of the runtime switching between the different configurations, but also because of the overhead associated with moving the configurations onto the servers and storing each such configuration in the server memory.
Performance degrades if the configurations are not preloaded in server memory as requests for different customer content or services arrive and implicate the different configurations at runtime. When a configuration implicated by an incoming request is not preloaded into server memory, the server conducts a relatively slow disk or network retrieval operation to retrieve the configuration. The server then loads the configuration into memory and reconfigures based on the configuration before it can respond to the request using the operational logic customized according to the implicated customer configuration.
Alternatively, the server can preload all customer configurations to memory. Even though each configuration consumes a small fraction of the server memory (e.g., kilobytes), the total memory allocated for customer configurations can snowball to gigabytes worth of memory when the platform supports thousands of customers, each with a custom defined configuration. The server reconfigures faster by having the customer configurations preloaded into memory than when having to retrieve them from storage or over a network. However, the lack of available memory may compromise other performance aspects of the server. For instance, the server may have a smaller cache footprint because of the larger proportion of memory allocated to preloading the customer configurations into memory. This can cause a greater number of cache misses which slows the ability of the server to respond to a greater number of requests.
Performance can also suffer in terms of server reload time. The servers periodically restart in order to load configurations that include updates or changes made by customers. In other words, restarting allows the server to remove stale configurations and resume operations with the newest or latest set of configurations. Reloading thousands of configurations into memory with each restart takes too much time, again degrading the server's performance in responding to incoming requests.
Also, many of the preloaded customer configurations are never accessed. Multi-tenant platforms typically perform a consistent distribution of customer content or services across the different servers. As a result, each server is tasked with a mere subset of all customer content or services. Therefore, preloading all customer configurations into memory would be wasteful operation as only a subset of the configurations are implicated on any server.
There is therefore a need for selective and efficient server reconfiguration with intelligent configuration management. More specifically, there is a need to dynamically modify the operational logic of a server at runtime in a multi-tenant platform without a one-to-one impact to memory for each supported configuration and without comprising runtime reconfiguration performance of the server, wherein the runtime reconfiguration performance is compromised from repeatedly retrieving configurations from disk or remote storage and loading the retrieved configurations into memory as the server receives requests directed to different customer content or services.